APPENDIX 6 



1 - The evaluation of the complexity of the reception filters 

Two cases have to be considered depending on whether it is sought or not to keep all 
the properties sought. Thus, for the 4n + 3 and 4n + 5 order filters: 

• To keep null ISI, the matched pair, the phase linearity of the sending and reception 

filters, as shown in Figures 10, four lattices are used. These four lattices work at the lowest 
rate. Given the fact that for each lattice two multiplications and additions are necessary, except 
for each first cell (one multiplier and one adder), the operational complexity is (8n + 4) MPU 
and APU, plus a sign inversion. 

• For a back-to-back system, simplifications are possible but only null ISI and the 

phase linearity of the sender filter are guaranteed. For the simplified 4n + 3 and 4n + 5 order 
reception filters, the operational complexity corresponds then to that of a single lattice plus that 
of the three adders, the multiplier by g and the inverter. The computations performed at the 
lowest rate can therefore be estimated at (2n + 3) MPU, (2n + 5) APU and one inversion. 

2 - A method of generic synthesis 

As compared with the initially proposed method of synthesis, we now have a synthesis 
method that can be used, for all three types of solutions, to meet a frequency specification such 
as the one shown in Figure 2. 

For the filter F(z) with a length L, different from 4n + 1, in other words for an TV order 

different from 4n and for a fixed value of the fall-back factor p, we seek to minimize the cost 
function: 



We use the FSQP (feasible sequential quadratic programming) algorithm developed by 
the team led by A.L. Tits [5] to search for the minimum of a set of non-linear even cost 
functions subjected to general, even and non-linear constraints. Since this method is a method 
of local optimization, the choice of an initial point is necessary. For a given weight w in the 
stopping band and a weight equal to 1 at the starting point and in the Nyquist frequency, 
namely 7i/8, we calculate an initial filter F init that is optimal for the norm of the minimal value. 
This filter F"" 7 is then used for direct computation of a set of lattice coefficients for a filter 



$(F) = su V {w P (l-\F(l)\)\w c P^-|F(e^)| , w s sup |F(e^)| 2 }. 
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producing a matched pair with null ISI. Since F"" 7 does not itself produce a matched pair with 
null ISI, the identification between the two sets of coefficients, the transversal coefficients of 
F /m7 and the lattice coefficients is not exactly achieved. We therefore obtain a different filter, 
referenced F"" 7 , whose lattice coefficients are considered to be the initial point of the 
optimization problem described by (63). For this problem, we fix w P = 1, w c = 2, w s =0.5. 

Furthermore, for any lattice structure optimized for a given value p 0 , we also use a 
continuation algorithm to obtain a lattice structure optimized for a value p ; , that is different in 
taking a sequence of intermediate values for p: the result of the lattice structure optimized for a 

value of p in the sequence is the initial point for the following value of p. 

Figure 22 shows a higher attenuation in the stopping band than could be obtained for a 
fixed length and fall-back factor. 

Let us consider for example the values L = 43 and p = 0.5: the best attenuation 

obtained by direct optimization is equal to 52.25 dB. The frequency response of the 
corresponding sending filter F°p 1 is given by Figure 23. The transversal coefficients and the 
coefficients of the lattice structure are given in Table 3. 



+ • 



Transversal coefficients 



5 



fo 


-1.033698 lO" 4 


/11 


8.074275371845 10~ 3 


fl 


-3.6954272174 10~ 4 


fl2 


1.5774357742194 10~ 2 


h 


6.60549905248 10" 4 


fis 


1.2603068512975 10~ 2 


h 


-1.09979829792 10" 4 


fu 


-6.358804372375 10~ 3 


U 


4.31563793685 10~ 4 




-3.3366501346017 10~ 2 


fs 


6.08998879046 10~ 4 


fie 


-4.6538674170018 10~ 2 


fe 


6.39123817678 10" 4 


fn 


-2.1044564354449 10~ 2 


h 


-7.82626716253 10" 4 


fis 


5.1214636348857 10~ 2 


h 


-3.761395765875 10~ 3 


/19 


1.51186813816588 10" 1 


h . 


-4.983816032565 10~ 3 




2.39358058463701 10" 1 


fio 


-9.24070721312 10" 4 


/21 


2.74720061562686 10" 1 



i2 



Lattice coefficients 



25 





-3.1848708937 




2.648518499 10" 1 


0:3 


-2.9385797132 


a 4 


-9.349457966 10" 1 


05 


9.246893837 10" 1 




-1.308356726 10" 1 


a- 


3.6683584963 




-6.126257549 10" 1 




-1.2465410017 


Q'10 


-1.439677875 10" 1 


a n 


-3.5749582735 



30 

Table 3 : Lattice and transversal coefficients of the example with L = 43 and p = 0.5 (f 42A = fj 



